Papers with synthetic data augmentation
Towards building a Robust Industry-scale Question Answering System (2020.coling-industry)
Copied to clipboard
| Challenge: | Existing systems that use “zero-shot transfer learning” (ZSTL) are difficult to train and have observation biases. |
| Approach: | They propose a production model called GAAMA which has two characteristics . it is robust and efficient, and trains on the recently introduced Natural Questions dataset . |
| Outcome: | The proposed model performs on two benchmarks: BioASQ and CovidQA. |
Sample, Translate, Recombine: Leveraging Audio Alignments for Data Augmentation in End-to-end Speech Translation (2022.acl-short)
Copied to clipboard
| Challenge: | End-to-end speech translation relies on data that pair source-language speech inputs with corresponding translations. |
| Approach: | They propose a method that augments transcriptions by sampling from suffix memory and translating them into target languages. |
| Outcome: | The proposed method delivers up to 0.9 and 1.1 BLEU points on top of augmentation with knowledge distillation on languages on CoVoST 2 and Europarl-ST. |
TADA : Task Agnostic Dialect Adapters for English (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work on dialectal English NLP is task-specific, using manual annotated dialect data, weak supervision, or data augmentation. |
| Approach: | They propose a method for task-agnostic dialect adaptation by aligning non-SAE dialects with task-specific adapters from SAE. |
| Outcome: | The proposed method improves dialectal robustness on 4 dialectal variants of the GLUE benchmark without task-specific supervision. |
Towards Robust Neural Retrieval with Source Domain Synthetic Pre-Finetuning (2022.coling-1)
Copied to clipboard
Revanth Gangi Reddy, Vikas Yadav, Md Arafat Sultan, Martin Franz, Vittorio Castelli, Heng Ji, Avirup Sil
| Challenge: | Existing neural IR systems rely on lexical matching for query-passage alignment, while masked language models use a dual encoder architecture to encode passages and questions into continuous vector representations. |
| Approach: | They propose to enhance the out-of-domain generalization of Dense Passage Retrieval (DPR) through synthetic data augmentation only in the source domain. |
| Outcome: | The proposed model outperforms existing models in in-domain and zero-shot evaluations on Wikipedia-based datasets. |
Visual Program Distillation with Template-Based Augmentation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Adapting visual programming to specialized tasks or domains remains challenging due to high annotation and inference costs. |
| Approach: | They propose a low-cost visual program distillation method that can be used for models with at most 1 billion parameters and requires no human-generated program annotations. |
| Outcome: | The proposed method can generate high-quality visual programs with no human-generated annotations with a relatively small amount of question/answer data. |
FiE: Building a Global Probability Space by Leveraging Early Fusion in Encoder for Open-Domain Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | generative models tend to be larger than extractive models due to the need for a decoder, run slower during inference due to auto-regressive decoded beam search, and their generated output suffers from hallucinations. |
| Approach: | They propose to extend transformer encoders with the ability to fuse information from multiple passages to provide cross-sample attention over all tokens across samples. |
| Outcome: | The proposed method outperforms the current state-of-the-art method by 2.5 Exact Match score on the Natural Question dataset while using only 25% of parameters and 35% of the latency during inference. |
Building A Proof-Oriented Programmer That Is 64% Better Than GPT-4o Under Data Scarcity (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing proof-oriented programming languages struggle due to data scarcity . existing proof-based models struggle due a lack of sufficient corpora for proof-orientated programming languages such as F* . |
| Approach: | They propose a method that synthesizes proof-oriented programming problems for proficiency in a language and incorporates diverse coding data for reasoning capability elicitation. |
| Outcome: | The proposed method outperforms existing proof-oriented models in function- and repository-level proof-based programming by 64% relative margin and improves GPT-4o's performance by 54% by repairing outputs over GPT-4) |